Apparatus and method for image processing

ABSTRACT

This disclosure relates to an image processing apparatus or an image processing method, and takes as its technical problem the fact that conventionally it is difficult to perform a process of carving out a rectangle with high precision in a noisy environment.  
     Therefore, as to an image made by binarizing an original image, histograms of the numbers of horizontal and vertical black pixels thereof are created and trapezoidal approximation is performed on the histograms so as to detect vertex coordinates indicating an extent of a rectangular area in the binary image. An inclination direction of the rectangular area in the extent is detected so that the rectangular area can be detected from the binary image with high precision.

BACKGROUND OF INVENTION

[0001] This invention relates to an image processing method and an imageprocessing apparatus, and more particularly, to an image processingmethod and image processing apparatus for detecting a rectangular image.

[0002] Various algorithms have been proposed so far for the sake ofdetecting and carving out a rectangular area from an original image. Forinstance, Japanese Patent Laid-Open No. 8-237537 has a technologydescribed wherein an outline is extracted first and the rectangular areadetected by looking for a portion of the outline constituting a straightline.

[0003] According to the conventional algorithms for carving out arectangle, however, it was difficult, in an environment of plenty ofnoise, to detect a straight line portion of a rectangular outline and tofurther detect one rectangle by associating four straight lines.Therefore, it was difficult to accurately carve it out so that precisionof the results of carving it out was not sufficient.

SUMMARY OF THE INVENTION

[0004] Accordingly, it is an object of the present invention to providean image processing method and an image processing apparatus capable ofdetecting a rectangular area with high precision even in an environmentof plenty of noise.

[0005] According to the patent invention, the foregoing object isattained by providing an image processing method for detecting arectangular area from an image, comprising: a histogram creation step ofcreating histograms of the numbers of horizontal and vertical blackpixels of a binary image made by binarizing an original image; arectangle extent detection step of performing trapezoidal approximationto the histograms so as to detect an extent of the rectangular area insaid binary image; and an inclination direction detection step ofdetecting an inclination direction of said rectangular area in said.

[0006] It is another object of the present invention to provide theimage processing method further comprising a labeling step of labelingsaid binary image, and wherein said histogram creation step creates thehistogram having the number of said labeled pixels.

[0007] Other features and advantages of the present invention will beapparent from the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings, which are incorporated in andconstitute a part of the specification, illustrate embodiments of theinvention and, together with the description, serve to explain theprinciples of the invention.

[0009]FIG. 1 is a block diagram showing an overview configuration of animage processing system according to an embodiment of the presentinvention;

[0010]FIG. 2 is a diagram showing an example of a binary image to bedetected according to the embodiment;

[0011]FIG. 3 is a diagram showing an example of a horizontal histogramof the number of black pixels according to the embodiment;

[0012]FIG. 4 is a flowchart showing a histogram calculation processaccording to the embodiment;

[0013]FIG. 5 is a flowchart showing a trapezoidal approximation methodof the histogram according to the embodiment;

[0014]FIG. 6 is a diagram showing an example of a detected coordinate ofa rectangular image according to the embodiment;

[0015]FIG. 7 is a diagram showing four bias filling modes according tothe embodiment;

[0016]FIG. 8 is a diagram for explaining a principle of determination ofa bias filling mode according to the embodiment;

[0017]FIG. 9 is a diagram showing an appearance of the histogram when aplurality of photos are placed according to a second embodiment;

[0018]FIG. 10 is a diagram for explaining a black block searching methodaccording to a third embodiment;

[0019]FIG. 11 is a flowchart showing an overview of a labeling processaccording to the third embodiment;

[0020]FIG. 12 is a flowchart showing a black block searching processaccording to the third embodiment;

[0021]FIG. 13 is a flowchart showing a filling process according to thethird embodiment;

[0022]FIG. 14 is a diagram showing a combined line segments fillingprocess according to the third embodiment;

[0023]FIG. 15 is a flowchart showing the combined line segments fillingprocess according to the third embodiment;

[0024]FIG. 16 is a flowchart showing a process of filling an area undercombined line segments according to the third embodiment;

[0025]FIG. 17 is a diagram for explaining the black block searchingmethod according to a fourth embodiment;

[0026]FIG. 18 is a flowchart showing the black block searching processaccording to the fourth embodiment;

[0027]FIGS. 19A and 19B are diagrams showing an example of a carved-outimage with insufficient rectangle detection precision;

[0028]FIG. 20 is a flowchart showing a background elimination processaccording to the embodiment;

[0029]FIG. 21 is a diagram showing an example of the horizontalhistogram of the number of white pixels at the left end of an image; and

[0030]FIG. 22 is a flowchart showing a process by a general image scanapplication.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Preferred embodiment of the present invention will now bedescribed in detail in accordance with the accompanying drawings.

[0032] Hereafter, each embodiment according to the present invention isdescribed in detail by referring to the drawings.

[0033] <First Embodiment>

[0034] System Configuration

[0035]FIG. 1 is a block diagram showing a configuration of a computersystem on which the present invention is implemented. Reference numeral101 denotes a CPU for controlling the entire system. Reference numeral102 denotes a keyboard used for data input to the system together with amouse of 102 a. Reference numeral 103 is a display unit comprised of aCRT, a liquid crystal and so on. Reference numeral 104 denotes an ROMand 105 denotes an RAM which constitute a storage unit of the system andstore a program to be executed by the system and data to be used by thesystem. Reference numeral 106 denotes a hard disk drive (HDD) and 107denotes a floppy disk drive (FDD) which constitute external storagecomponents to be used for a file system of the system. Reference numeral108 denotes a printer.

[0036] Reference numeral 109 denotes a network I/F connected to an LANor a WAN.

[0037] It is possible, via the I/F 109, to exchange a data file such asa computer program or a digital image with another computer. The programand data are recorded on the hard disk 106 or directly rendered to theRAM 105 and executed.

[0038] Reference numeral 110 denotes an image scanner capable of readinga photo and printing on a document glass and capturing them as amultivalic digital image onto the RAM 105.

[0039] Overview of the Rectangular Image

[0040]FIG. 2 shows an example of the image captured from the imagescanner 110 and binarized. The rectangle in the drawing represents arectangular manuscript image (hereafter, referred to as a rectangularimage) such as a photo, and three ellipses represent noise. While thisdrawing shows the noise simplified for the sake of explanation, thenoise in reality is of indeterminate size and shape, and the numberthereof is often a few hundred to a few thousand pieces. As forbinarization of the image, a known simple binarization method by meansof comparison with a predetermined threshold is used, and a detaileddescription thereof is omitted here.

[0041] The image shown in FIG. 2 is held as a bitmap W pixels wide and Hpixels high on the RAM 105. Each pixel is 1 byte deep, and a black pixelis indicated by a value 255 while a white pixel is indicated by a value0. The bitmap on the RAM 105 is accessible as a two-dimensional array,where the pixel located horizontally at an i-th position and verticallyat a j-th position starting from the upper left of the bitmap isexpressed as buf(i,j). According to this embodiment, a coordinate of thearea in which the rectangular image exists is identified from thisbitmap by the following method.

[0042] Histogram Creation

[0043]FIG. 3 is a diagram for, when assuming an ideal state of no noiseexisting in the binary image shown in FIG. 2, showing an appearance of ahistogram of the number of black pixels counted horizontally, and theupper part of the diagram shows the binary image and the lower partshows the example of the histogram. It is understandable from thisdrawing that the histogram approximately forms a trapezoid bycorresponding to four vertexes of the rectangle of the binary image.While the histogram will not be a perfect trapezoid since the binaryimage in reality includes the noise, it is possible, from thetrapezoidal shape of this histogram, to obtain sufficient information toidentify the positions of the horizontal vertexes.

[0044]FIG. 4 is a flowchart showing a creation process of the histogramshown in FIG. 3. Hereafter, an array for storing histogram data isindicated as hist( ).

[0045] First, 0 is assigned to each of variables x and y in steps S401and S402, and 0 is assigned to a variable hist(x) in a step S403.

[0046] A pixel buf(x,y) on the bitmap is compared to the value 255 in astep S404. If the pixel and the value 255 are different, the processproceeds to a step S406. If they are equal, the process moves on to astep S405 and increments hist(x) by just a value 1, and then moves on tothe step S406.

[0047] The variable y is incremented by just a value 1 in the step S406and is compared to the bitmap height H next in a step S407 next. If yand H are different, the process returns to the step S404. If equal, theprocess moves on to a step S408 and increments the variable x by just 1,and compares x to the bitmap width W in a step S409. If x and W aredifferent, the process returns to the step S402. If equal, the processterminates.

[0048] Vertex Detection Process

[0049] The array hist( ) of the histogram of the number of black pixelscounted can be obtained by the above process. As shown in FIG. 3, thehistogram forms the trapezoid surrounded by an X-axis and three straightlines. As for the trapezoid in this embodiment, the position at whichthe left side crosses the X-axis is a, the position at which the leftside crosses the upper base is b, the position at which the right sidecrosses the upper base is c, and the position at which the right sidecrosses the X-axis is d.

[0050] Here, a description is given using the flowchart in FIG. 5 of amethod of seeking the positions a, b by analyzing the left side of thetrapezoid presented by the histogram.

[0051] First, in a step S501, a peak point P at which hist(x) is maximumis detected. In the histogram shown in FIG. 3, there is a peak of height800 around X=400. Next, in a step S502, a point F of half the height ofthe peak is detected forward of the point P. In the histogram shown inFIG. 3, the position before or after X=100 is equivalent to the point F.And a point F0 of ¼ of the height of the peak is detected forward of thepoint F in a step S503. Furthermore, a point F1 of ¾ of the height ofthe peak is detected backward of the point F in a step S504.

[0052] Next, in a step S505, linear approximation is performed by usinga known regression analysis as to all the points of F0≦X≦F1 so as toobtain a straight line L as shown in FIG. 3. The point at which thestraight line L crosses the X-axis is calculated in a step S506, whichpoint is obtained as a. The point at which the straight line L becomes98 percent of the height of the peak P is calculated in a step S507,which point is obtained as b. This is for the purpose of consideringthat the peak P includes the noise and is more than the actual peak,correcting it to be 98 percent.

[0053] The foregoing description with reference to the flowchart shownin FIG. 5 pertains to an example of performing a search forward of thepeak P and thereby obtaining the points a, b. Likewise, the points c, dcan be obtained by performing a search backward of the peak P.

[0054] Moreover, while the foregoing description pertains to horizontalvertex position detection, it is possible to perform vertical vertexposition detection by reversing vertical and horizontal axes in thedescription and applying exactly the same method. It is thus possible,as shown in FIG. 6, to obtain four coordinate values, vertical andhorizontal, surrounding the rectangle respectively, that is, eightvariables (ax, bx, cx, dx, ay, by, cy and dy) as vertex candidates.

[0055] Here, the eight variables are uniquely determined from therectangle, yet the rectangle cannot be inversely determined from theeight variables. To be more specific, four types of vertex coordinatescan be generated from these eight variables according to bias fillingdirections (hereafter, referred to as bias filling modes) thereof asshown in FIG. 7, so the correct rectangle cannot be uniquely determined.In FIG. 7, a special case (not shown) where the rectangle is completelyhorizontal is a mode 0, and the four bias filling modes are modes 1, 2,−1 and −2 respectively. As histograms of the same shape are generatedfrom the images shown in these modes, it is necessary to determine thebias filling modes by a method other than the histogram.

[0056] Bias Filling Mode Determination

[0057] Hereafter, a bias filling mode determination process according tothis embodiment is described in detail.

[0058] While the above-mentioned FIG. 6 is equivalent to the state ofthe mode 1 shown in FIG. 7, it is considered that, in the case ofcounting the black pixels existing inside two triangular areas shown bydiagonal lines in FIG. 6, the results of counting the black pixelsbecome 0 because ideally the black pixels (dotted portion in thedrawing) are not included. Hereafter, the areas shown by the diagonallines shown in FIG. 6 are called the template of the mode 1. Likewise,as for the other modes shown in FIG. 7, the two triangular areasexisting in the upper portion of the rectangle are the template of eachmode.

[0059] Here, as shown in FIG. 8, if the template (the portion ofdiagonal lines in the drawing) of the mode 2 is applied to therectangular image (dotted portion in the drawing) which is in realitymode 1, the rectangular portion extends into the inside the template, sothe black pixel count does not become 0. According to this embodiment,this characteristic is utilized to determine the bias filling mode ofthe rectangle.

[0060] To be more precise, counting is performed as to the numbers ofblack pixels in the templates in the case of applying the templates ofthe four modes respectively, and the template having the smallestresulting black dot count is determined as the bias filling mode of thatimage. To be more specific, the smallest degree of intrusion into thetemplate is the degree of the bias filling at which the rectangularimage becomes most approximate. As a matter of course, it is alsopossible to count the number of black pixels existing outside thetemplates so as to determine the bias filling mode according to thelargest resulting black dot count.

[0061] The above processing can identify the four vertex coordinates ofthe rectangular area in the binary image. Therefore, it is possible, byperforming known bias filling correction and image carving-out processesto the original image based on this information, to carve out arectangular photo image placed on the image scanner 110 for instance.

[0062] As for the known bias filling correction process, it is desirableto use simple interpolation (nearest neighbor method) or linearinterpolation (bilinear method), which should be chosen according to useconsidering that the former is speedier and the latter provides betterimage quality.

[0063] As described above, it is possible, according to this embodiment,to detect the rectangular area precisely even in the environment ofplenty of noise.

[0064] If the rectangular area is a perfect rectangle, the trapezoid ofwhich right and left hypotenuses are symmetric should be presented as anideal histogram. However, there are the cases in which, due to theinfluence of the noise, the right and left hypotenuses are notsymmetric, that is, the distance between a and b shown in FIG. 3 is notequal to the distance between c and d. In such cases, trapezoidalapproximation is performed according to the data obtained from thehypotenuse considered to be less influenced by the noise. To be morespecific, precision (square error between the straight line and actualdata) of the linear approximation by the regression analysis should becompared as to the right and left hypotenuses so as to adopt the onehaving a less error.

[0065] Likewise, it is also possible that the rectangular form linkingthe four vertexes obtained as a result of rectangle detection does notnecessarily become the rectangle due to the influence of the noise andadjacent sides are not orthogonal. In such cases, a correction should bemade to the rectangular form so that it becomes the rectangle based onthe data presumably less influenced by the noise, and thus the precisionof the rectangle detection can be improved.

[0066] <Second Embodiment>

[0067] Hereafter, a second embodiment of the present invention isdescribed. The system configuration of the second embodiment is the sameas that of the above-mentioned first embodiment, and so a descriptionthereof is omitted.

[0068] A case in which one rectangular area exists in the image wasdescribed in the above-mentioned first embodiment. As for such asituation, for instance, an assumption can be made about the processwherein a scan is performed in a state of not necessarily horizontallyplacing a photo on the document glass of the image scanner 110, and thephoto image is extracted thereafter by automatically correcting the biasfilling and carving out the image.

[0069] However, the document glass of a general flat head scanner allowsplacement of a document of A4 size or so, and so a plurality of generalphotos (so-called L-version size) can be placed thereon at once.Therefore, scanning can be rendered more efficient if it is possible toread the plurality of photo images by scanning once and carving out onlythe rectangular areas showing the photo images from the obtained images.

[0070] Thus, the second embodiment is characterized by detecting aplurality of rectangular images from one image.

[0071] Hereafter, a description is given by referring to FIG. 9 of theprocess of detecting a plurality of rectangular images according to thesecond embodiment. The left portion of FIG. 9 shows a binarized image onseparately placing two photos on the document glass in a verticalscanning direction (longitudinal direction in the drawing), and theright portion thereof shows the appearance of the histogram in thevertical scanning direction of the binarized image. As is clear from thedrawing, in the case in which a plurality of photos are separatelyplaced, a plurality of trapezoids appear on the histogram. Therefore,according to the second embodiment, this histogram is referred to so asto cut the image into two between the plurality of trapezoids, that is,at the position indicated by a dashed line L in FIG. 9. Then, it ispossible thereafter, by the method shown in the above-mentioned firstembodiment, to detect the rectangular images in each of the cut areasrespectively.

[0072] The case of separately placing the plurality of photos in thevertical scanning direction was described as to FIG. 9. Likewise, italso is possible, in the case of separately placing them in a horizontalscanning direction (latitudinal direction in the drawing) to cut theimage in the horizontal scanning direction and extract each rectangularimage by creating the histogram as to the horizontal scanning direction.

[0073] As described above, according to the second embodiment, it ispossible to detect a plurality of rectangular images from one image.

[0074] <Third Embodiment>

[0075] Hereafter, a third embodiment of the present invention isdescribed. The system configuration of the third embodiment is the sameas that of the above-mentioned first embodiment, and so a descriptionthereof is omitted.

[0076] An image processing method according to the third embodiment ischaracterized by, as to the binary image shown in FIG. 2, detecting as ablack block a predetermined-size block having the number of the blackpixels exceeding a predetermined ratio therein and replacing the blackpixels in the detected block and those coupled thereto with apredetermined label value so as to eliminate the influence of the noiseand detect the rectangular images with higher precision.

[0077] Here, an ideal labeling process of the binary image shown in FIG.2 is, based on mutual connections of the black pixels, to give a pixelvalue 1 to every pixel included in the upper rectangular object and givea pixel value 2 to every pixel included in the lower rectangular object,for instance. To be more specific, a label is not given to a minutenoise portion so as to leave it as-is at the pixel value 255.

[0078] Labeling Process Overview

[0079] Hereafter, a labeling method for implementing the ideal labelingis described in detail.

[0080] A variety of so-called labeling algorithms for labeling eachobject of the binary image have been proposed so far. However, a generalbinary image includes a minute noise area that is not the object andthus does not primarily need to be labeled, as a matter of course.Therefore, also from the viewpoint of processing time, it is desirableto exert control so as to allocate no label to the minute noise area inthe image.

[0081] To implement the control, for instance, it is possible toeliminate known isolated points as preprocessing of the labeling,thereby decreasing the noise in the image and avoiding wasteful labelingso as to speed up the labeling.

[0082] However, even if the isolated points are eliminated as thepreprocessing of the labeling, it has not been possible to eliminate thenoise not completely isolated and having a few pixels or so connectedthereto. Therefore, the problem of allocating a label to the minutenoise area has not been completely solved.

[0083] As for a labeling speedup method, the method described inJapanese Patent Laid-Open No. 2000-285240, for example, has beenproposed. According to this method, a general filling algorithm is usedto fill a connection area of the black pixels, and then one label isallocated to the filled area. The filling and label allocation processesare repeated so that the labeling is performed to all the images. Thismethod is characterized in that, when the number of the filled pixels issmall, the area is not labeled and the process is skipped since it issupposedly an isolated and minute noise area. This characteristic isinstrumental in eliminating redundant label allocation.

[0084] However, even the labeling method described in Japanese PatentLaid-Open No. 2000-285240 requires the process of filling the connectionarea to be performed to all the black pixels. Therefore, there is alimit to the speedup of the labeling, particularly in the case of animage having plenty of noise.

[0085] Thus, an object of the third embodiment is to provide an imageprocessing apparatus for implementing a high-speed labeling process inwhich the effect of noise has been eliminated, and the method thereof,so as to solve the problem of the conventional art.

[0086]FIG. 10 is a diagram for explaining a method of searching for ablock which is corresponding to the upper left portion including therectangular object in the upper part in FIG. 2 and is a starting pointof the filling process of the third embodiment.

[0087] First, as shown in FIG. 10, the entire bitmap of W pixels×Hpixels shown in FIG. 2 is divided into blocks of w pixels×h pixels, andthe block having its inside completely occupied by the black pixels ofthe value 255 (hereafter, referred to as “black block”) is searched forin the direction of the arrow in the drawing. The block shown as a blackrectangle in FIG. 3 is the discovered black block.

[0088] According to the third embodiment, high-speed labeling isimplemented by filling the area in which the black pixels are connectedto the discovered black block. FIG. 11 shows a flowchart of the entirelabeling process according to the third embodiment, and a descriptionthereof is given.

[0089] First, the value 1 is assigned to the variable c in a step S1401,and a black block search is performed in a step S1402. The black blocksearch process is described in detail later.

[0090] Next, in a step S1403, it is determined whether or not a blackblock is discovered. If discovered, the process moves on to a stepS1404. If not discovered, the process ends.

[0091] In step S1404, the black pixel connection area is filled with thelabel c starting from the discovered black block. The filling process isdescribed in detail later.

[0092] The variable c is incremented by just 1 in a step S1405, and thenthe process returns to a step S1402 to repeat the above processing.

[0093] As a result of this processing, it is possible to provide thelabels 1, 2, . . . , c only to those objects which are of apredetermined size or larger.

[0094] Black Block Search Process

[0095] Hereafter, the flowchart in FIG. 12 shows the details of theblack block search process shown in the step S1402 in FIG. 11, and adescription thereof is given.

[0096] First, variables y, x, j and i are initialized with the value 0in steps S1501 to 1504 respectively. In a step S1505, it is determinedwhether or not buf(x+i, y+j) is 255 indicating the presence of a blackpixel. If it is 255, the process proceeds to a step S1506. If not, theprocess moves on to a step S1510.

[0097] The variable i is incremented by just 1 in a step S1506. If i isequal to w indicating a block width in a step S1507, the process moveson to a step S1508. If different, the process moves on to step S1505.

[0098] The variable j is incremented by just the value 1 in the stepS1508. If j is equal to h, indicating a block height in a step S1509,the process ends. At this time, the block of w×h pixels, of which upperleft starting point is (x, y), is detected as the black block. If j isdifferent from h, the process returns to the step S1504.

[0099] In the step S1510, w is added to x. In a step S1511, if x isequal to or larger than W indicating an image width, the processproceeds to a step S1512. If not, the process returns to the step S1503.

[0100] In the step S1512, h is added to y. In a step S1513, if y isequal to or larger than H indicating an image height, the processterminates, and at this time, it is determined that no black blockexists. If y is smaller than H, the process returns to the step S1502.

[0101] The black block of w×h pixels is detected from the binary imageof W pixels×H pixels by the above processing.

[0102] Connection Area Filling Process

[0103] The flowchart depicted in FIG. 13 shows the details of theconnection area filling process shown in the step S1404 in FIG. 11, adetailed description of which is now given.

[0104] Various methods are known as an object filling process, and anyfilling method may be used according to the third embodiment. Therefore,a basic method is described here, and it is assumed for the sake ofsimplified description that the object is a simple shape which isexternally convex.

[0105] As a result of the above-mentioned black block search process,the block of which the upper left starting point is (x, y) is detectedas the black block as shown in FIG. 14. FIG. 14 is a diagram showing acombined line segments filling process according to the thirdembodiment. Thus, in the filling process according to the thirdembodiment, the area connected with the black pixels is filled with thelabel c from ((x+w)/2, (y+h)/2) which is the center point of the blackblock as the starting point. To be more specific, the pixel value whichis the value 255 is replaced by the value c.

[0106] That is, first, the black pixel connection area on a scan lineincluding the center point ((x+w)/2, (y+h)/2) of the black block isfilled with the label c in a step S601. Thus, filling with the label cis performed to the line segment (line segment for horizontally cuttingthe object in FIG. 14) connecting the left end (x0, y+h/2) to the rightend (x1, y+h/2) of the scan line in the object including the black blockin FIG. 14. The line segment connecting (x0, y+h/2) to (x1, y+h/2) isreferred to as a connection line segment hereafter, and the fillingprocess thereof is described in detail later.

[0107] The black pixel connection portion from the connection linesegment downward is filled in a step S602, and the black pixelconnection portion from the connection line segment upward is filled ina step S603. The filling process in the steps S602 and S603 is describedin detail later.

[0108] As a result of this process, it is possible to fill with thelabel c the area connected to the black block by the black pixels.

[0109] Connection Line Segment Filling Process

[0110] Hereafter, the flowchart in FIG. 15 shows the details of theconnection line segment filling process shown in the step S601 in FIG.13, and a description thereof is given. To simplify the description, thecenter point ((x+w)/2, (y+h)/2) of the black block which is the startingpoint of the process is indicated by coordinates (X, Y). That is,X=(x+w)/2, Y=(y+h)/2.

[0111] First, X is assigned to the variable i in a step S801, and i isdecremented by the value 1 in a step S802. It is determined whether ornot buf(i, Y) is equal to the value 255 in a step S803, and ifdifferent, the process proceeds to a step S804. If equal, the processreturns to the S802 to further decrement i.

[0112] A value i+1 is stored in a variable x0 in the step S804, and X isassigned to the variable i in a step S805, i is incremented by 1 in astep S806, It is determined whether or not buf(i, Y) is equal to thevalue 255 in a step S807. If they are equal, the process proceeds to astep S808. If different, the process returns to the S806 to furtherincrement i.

[0113] A value i−1 is stored in a variable x1 in the step S808, a valuex0 is assigned to the variable i in a step S809, and c is assigned tobuf(i, Y) in a step S810. And i is incremented by just 1 in a step S811.The process terminates if i is larger than x1 in a step S812. If not,the process returns to the step S810.

[0114] It is possible, by this process, to store x coordinates at bothends of the connection line segment in the variables x0, x1 and fill theconnection line segment with the label c.

[0115] Filling Process From the Connection Line Segment Downward(Upward)

[0116] The flowchart in FIG. 16 details the filling process from theconnection line segment downward shown in the step S602 in FIG. 13, anda description thereof is given. It is also Y=y+h/2 here, and so theblack pixel connection portion from the connection line segmentconnecting (x0, Y) to (x1, Y) downward is filled with the label c.

[0117] First, the variable Y is incremented by just 1 in a step S901,and x0 is assigned to the variable i in a step S902. It is determinedwhether or not buf(i, Y) is equal to the value 255 in a step S903, andif different, the process proceeds to a step S904. If equal, the processproceeds to a step S906.

[0118] The variable i is incremented by just 1 in the step S904. If i islarger than x1 in a step S905, the process terminates. If not, theprocess returns to the step S903.

[0119] In a step S906, i is assigned to X, and the connection linesegment which is horizontal, that is, on the scan line, is filledstarting from (X, Y) in a step S907. The filling process on the scanline is shown in the flowchart in FIG. 15, and so a detailed descriptionthereof is omitted here.

[0120] As for the filling process of the black pixel connection portionfrom the connection line segment upward shown in the step S603 in FIG.13, it is basically the same as the downward filling process shown inthe flowchart in the above-mentioned FIG. 16, where Y is decrementedinstead of being incremented and the same processing is performed in theother steps.

[0121] According to the labeling process described above, it is possibleto label only the objects that include predetermined-size black blocks,that is, the objects which have a predetermined size or larger area.Therefore, no minute noise area is labeled and so that high-speedlabeling becomes possible.

[0122] Rectangle Detection After Labeling

[0123] According to the above labeling process, in the two-dimensionalarray of buf() shown in FIG. 2, the pixel value 1 as a label 1 is storedfor every pixel included in the rectangular object in the upper portionof the drawing, the pixel value 2 as a label 2 is stored for every pixelincluded in the rectangular object in the lower portion of the drawing,the pixel value 255 is stored for the black pixel in any other minutenoise area, and the value 0 is stored for the white pixel in thebackground respectively.

[0124] According to the third embodiment, the position of therectangular object (vertex coordinate) is determined for each labelarea. To be more specific, the rectangle detection process described inthe above-mentioned first and second embodiments is performed on theimage after the labeling. That is, in the histogram creation flowchartshown in the above-mentioned FIG. 4, the comparison subject of buf()shown in the step S404 is changed from the value 255 indicating theblack pixel to a label value (1 or 2 in this case). The areas of thelabel 1 and label 2 are thereby detected as the rectangles.

[0125] As can be appreciated by those of ordinary skill in the art,although the foregoing description assumes two rectangles as an example,it is of course possible to perform the above labeling process on abinary image including an arbitrary number of rectangles, performing therectangle detection process for each label.

[0126] As described above, according to the third embodiment, it ispossible to eliminate the effect of noise and detect a rectangular imagewith higher precision by performing the ideal labeling process to thebinary image.

[0127] <Fourth Embodiment>

[0128] Hereafter, a fourth embodiment of the present invention isdescribed. The system configuration of the fourth embodiment is the sameas that of the above-mentioned first embodiment, and so a descriptionthereof is omitted.

[0129] According to the above-mentioned third embodiment, when searchingfor the black block in the binary image, the entire binary image isdivided into the blocks and then it is checked whether or not all thepixels inside each block are black pixels. However, there are also othermethods that can be used as the black block search method. For instance,in the fourth embodiment, a method of successively changing the upperleft coordinates of the blocks in the binary image is described.Although this method increases the calculation load, it improvesperformance of detecting the black blocks.

[0130]FIG. 17 is a diagram showing an instance of searching for theblack block in the rectangular object according to the fourthembodiment. According to the drawing, it is understandable that thestarting point (x, y) of the search according to the fourth embodimentis located out of the block boundary shown in FIG. 10 according to thethird embodiment.

[0131]FIG. 18 shows a flowchart of the black block searching processaccording to the fourth embodiment, which is described in detail.

[0132] First, variables y, x, j and i are initialized with the value 0in steps S1101 to S1104 respectively. It is determined whether or notbuf(x+i, y+j) is 255 indicating the black pixel in a step S1105. If itis 255, the process proceeds to a step S1106. If not, the processproceeds to a step S1110.

[0133] The variable i is incremented by just 1 in a step S1106. If i isequal tow indicating the block width in a step S1107, the processproceeds to a step S1108. If different, the process proceeds to the stepS1105.

[0134] The variable j is incremented just by the value 1 in the stepS1108. If j is equal to h indicating the block height in a step S1109,the process terminates, and at this time, the block of w×h pixels ofwhich the upper left starting point is (x, y) is detected as the blackblock. If j is different from h, the process returns to the step S1104.

[0135] In a step S1110, i+1 is added to x. If x is incremented merely bythe value 1 here, exactly the same results can ultimately be obtained.However, i+1 is added so as to exclude a wasteful search and speed upthe processing.

[0136] If x is equal to or larger than W indicating the image width in astep S1111, the process proceeds to a step S1112. If not, the processreturns to the step S1103.

[0137] In a step S1112, y is incremented by just the value 1. In a stepS1113, if y is equal to H, indicating the image height, the processterminates. At this time, it is determined that no black block exists.If y is not equal to H, the process returns to the step S1102.

[0138] The black block of w pixels×h pixels is detected from the binaryimage of W pixels×H pixels by the above processing. Thereafter, theblack pixel connection area filling and rectangle detection processescan be performed as in the third embodiment.

[0139] As described above, according to the fourth embodiment, it ispossible to detect the black block of predetermined size more precisely.Therefore, labeling can be performed only to an effective object withhigher precision, and consequently more precise rectangle detectionbecomes possible.

[0140] According to the above-described third and fourth embodiments,all the pixels in the block should be black pixels as the condition fordeclaring the block a black block. However, there are cases in which alarge number of white pixels get mixed inside the black pixel areadepending on the condition of the noise. Thus, it is also possible,according to the present invention, to count the number of black pixelsinside the block so that, if the block exceeds the predeterminedthreshold, the block is determined to be a black block.

[0141] The black block to be searched for does not necessarily have tobe a rectangle. As can be appreciated by those of ordinary skill in theart, the black block may instead be a polygon such as a hexagon, acircle or an ellipse, or it may be selectable according to the shape ofthe assumed object.

[0142] It is also possible, by not limiting the size of the search blockto one predetermined size, to start the search from a large block size,such that, for example, if the object is not discovered, the search isrepeated by gradually reducing the size. Thus, it is possible to avoidthe danger of mistakenly taking an object smaller than the assumed onefor the noise and ignoring it.

[0143] <Fifth Embodiment>

[0144] Background Elimination

[0145] An overview of a typical rectangle carving-out process isdescribed by referring to FIGS. 19A and 19B. First, as for FIG. 19A,consideration is given to the case of carving out the rectangularportion indicated by dots from the image surrounded by solid lines. Inthis case, if the precision of carving out is low, a slightly deviatedrectangle is carved out as shown by the dashed lines in this drawing.FIG. 19B shows an appearance of such deviated carving-out results havingthe bias filling correction performed thereto. It is understandable fromthis drawing that background areas are inside the four sides of thecarving-out results like wedges.

[0146]FIGS. 19A and 19B schematically shows a typical example of therectangle carving-out, where the four sides do not necessarily becomesymmetric, and it is also likely that the background is inside only oneside.

[0147] Therefore, with the conventional carving-out process, in general,as shown by the dashed lines in FIG. 19B, the background was notincluded in the carving-out results by carving out the rectangle furtherinside than the detection results indicated.

[0148] However, with this conventional carving-out process, as shown inFIG. 19B, there was a possibility of losing a part of the information onthe foreground to be primarily carved out by narrowing the carving-outarea so as not to include the background in the carving-out results.

[0149] Thus, an object of a fifth embodiment is to provide the imageprocessing method and image processing apparatus capable of minimizingcutoff of the background mixed area in the rectangular image carved outof the original image in order to solve the above-mentioned problem.

[0150] Another object is to provide an image processing method and imageprocessing apparatus capable of efficiently detecting the rectangularimage even in a noise-rich environment.

[0151] Hereafter, a background elimination process according to thisfifth embodiment is described using the flowchart shown in FIG. 20,taking the left end of the image as an example.

[0152] First, in a step 2001, the carved-out image is binarized by usinga known simple binarization method. In this case, the binarizationthreshold can be set arbitrarily by a user via a predetermined UI and soon.

[0153] In a step 2002, a histogram of the number of the white pixels iscreated by the same method as that of creating the histogram of thenumber of the black pixels described with reference to FIG. 4. It shouldbe noted that, although a histogram of the entire image was created bythe method in FIG. 4, it is sufficient here to create one of only theproximate area of the left end of the image. A subject area for creatingthis histogram may be set up appropriately according to the system. Forexample, if the size of the carved-out rectangular image is w pixels×hpixels, it is sufficient to create a histogram of the area of w/20pixels or so from the left end. FIG. 21 shows an example of the createdhistogram. In FIG. 21, the upper part is the carved-out binary image,and the lower part is the histogram of the white pixels.

[0154] In a step 2003, the area in which the white pixels are mixed isdetermined in the carved-out image, and the right end of the white pixelmixed area is indicated as a straight line L. As is clear from thehistogram shown in FIG. 21, the histogram gets close to 0 around theportion in which the white pixels mixed at the left end of thecarved-out image run out. The histogram does not completely become 0 dueto the white pixels mixed at the top of the image and the influence ofthe noise. This embodiment is characterized by utilizing such acharacteristic of the white pixel histogram to cut off the white pixelarea mixed in the carved-out image. To be more precise, in FIG. 21, thearea from the left end of the histogram to the position at which theheight of the histogram becomes small enough compared to the height ofthe image h (about 3 percent of h, for instance) is determined as thewhite pixel mixed area, and the right end of the white pixel mixed areais indicated as the straight line L. As a determination threshold of thestraight line L (3 percent in this example) depends on the performanceof the scanner and soon, it is also possible to have the threshold setup arbitrarily by the user.

[0155] Next, in a step 2004, the white pixel mixed area is cut off fromthe carved-out image. To be more specific, in FIG. 21, the left end ofthe carved-out image is cut off by the straight line L so as toeliminate the background at the left end of the image.

[0156] As described above, it is possible to eliminate the white pixelmixed area, that is, background pixels as to the four sides of thecarved-out image by repeating the background elimination process for theleft end of the image described by using FIG. 20 likewise to the rightend, top and bottom thereof.

[0157] As described above, according to this embodiment, it is possibleto precisely cut off the white pixel mixed area at the end of the imagecaused by insufficient carving-out precision of the scanned rectangulararea, thus allowing the rectangular area to be efficiently detectedwithout narrowing the carving-out area, even in a noise-richenvironment.

[0158] <Sixth Embodiment>

[0159] Hereafter, a sixth embodiment of the present invention isdescribed. The system configuration of the sixth embodiment is the sameas that of the above-mentioned fifth embodiment, and so a descriptionthereof is omitted.

[0160] The above-mentioned fifth embodiment considers an example whereinthe position of the rectangle is automatically determined, it is carvedout, the bias filling is corrected and thereafter the white pixel mixedarea at the end of the image is cut off. However, the present inventionis not limited to such an instance, and it is also possible to cut offthe white pixel mixed area after performing a general manual rectangularscan.

[0161] Here, a description is given by referring to the flowchart inFIG. 22 as to the operation in the case of having a general image scanapplication executed on the system shown in FIG. 1. First, in a stepS2201, the entire surface of the document glass of the scanner 110 isscanned at a low resolution of 75 dpi or so, and the obtained bitmapdata is stored in the RAM 105. The low-resolution bitmap is thendisplayed on the display unit 103 in a step S2202, and the userspecifies a desired rectangle (a photo area, for instance) on thedisplayed bitmap by using the keyboard 102, mouse 102 a and so on in astep S2203. The specified rectangular area is scanned at a highresolution in a step S2204, and next, manual trimming is furtherperformed to the obtained high-resolution rectangle data by using themouse 102 a and so on in a step S2205.

[0162] As described above, according to the scan procedure having beenperformed so far, the rectangle selection is performed based on alow-resolution preview, and then the high-resolution scan and trimmingare performed. As for this method, precision of the position is notsufficient because the rectangle selection is performed based on thelow-resolution preview, and so trimming work in a step S2205 isessential. Otherwise, mixture of the white pixels is accepted, or anarea sufficiently inside the original rectangle is specified whenspecifying the rectangle in the step S2203.

[0163] Thus, according to the sixth embodiment, the manual trimmingprocess shown in the step S2205 is replaced by the backgroundelimination process described by using FIGS. 20 and 21 in theabove-mentioned fifth embodiment. The trimming process can thereby beautomated.

[0164] As described above, according to the sixth embodiment, it ispossible to improve the quality of the rectangle data such as thescanned photo image while using the general scan application.

[0165] As described above, according to the fifth and sixth embodiments,it is possible to keep the cutoff of background mixed area minimum inthe rectangular image carved out of the original image.

[0166] Therefore, it is possible to detect the rectangular imageefficiently even in the environment of plenty of noise.

[0167] The present invention may be applied to either a systemconstituted by a plurality of devices (e.g., a host computer, interfacedevice, reader, printer, and the like), or an apparatus consisting of asingle equipment (e.g., a copying machine, facsimile apparatus, or thelike).

[0168] The objects of the present invention are also achieved bysupplying a storage medium (or recording medium), which records aprogram code of a software program that can implement the functions ofthe above-mentioned embodiments to the system or apparatus, and readingout and executing the program code stored in the storage medium by acomputer (or a CPU or MPU) of the system or apparatus. In this case, theprogram code itself read out from the storage medium implements thefunctions of the above-mentioned embodiments, and the storage mediumwhich stores the program code constitutes the present invention. Thefunctions of the above-mentioned embodiments may be implemented not onlyby executing the readout program code by the computer but also by someor all of actual processing operations executed by an operating system(OS) running on the computer on the basis of an instruction of theprogram code.

[0169] Furthermore, the functions of the above-mentioned embodiments maybe implemented by some or all of actual processing operations executedby a CPU or the like arranged in a function extension card or a functionextension unit, which is inserted on or connected to the computer, afterthe program code read out from the storage medium is written in a memoryof the extension card or unit.

[0170] When the present invention is applied to the storage medium, thatstorage medium stores program codes corresponding to at least one of theflow charts shown in FIGS. 4, 5, 11, 13, 15, 16, 18, 20 and 22 mentionedabove.

[0171] As many apparently widely different embodiments of the presentinvention can be made without departing from the spirit and scopethereof, it is to be understood that the invention is not limited to thespecific embodiments thereof except as defined in the appended claims.

What is claimed is:
 1. An image processing method for detecting arectangular area from an image, comprising: a histogram creation step ofcreating histograms of the numbers of horizontal and vertical blackpixels of a binary image made by binarizing an original image; arectangle extent detection step of performing trapezoidal approximationto the histograms so as to detect an extent of the rectangular area insaid binary image; and an inclination direction detection step ofdetecting an inclination direction of said rectangular area in saidextent.
 2. The image processing method according to claim 1, whereinsaid rectangle extent detection step performs the trapezoidalapproximation to each of said horizontal and vertical histograms of saidbinary image, and detects the extent of the rectangular area in each ofthe horizontal and vertical directions of said binary image.
 3. Theimage processing method according to claim 2, wherein said rectangleextent detection step detects a plurality of vertex coordinatecandidates for said rectangular area.
 4. The image processing methodaccording to claim 3, wherein said rectangle extent detection stepdetects vertex coordinates of said approximated trapezoid as theplurality of vertex coordinate candidates for said rectangular area. 5.The image processing method according to claim 4, wherein saidinclination direction detection step identifies coordinates of fourvertexes from the plurality of vertex coordinate candidates for saidrectangular area.
 6. The image processing method according to claim 5,wherein said inclination direction detection step identifies said fourvertex coordinates by: hypothesizing as to a plurality of inclinationdirections of said rectangular area based on the plurality of vertexcoordinate candidates for said rectangular area; counting the numbers ofblack pixels existing outside the rectangular area indicated by each ofthe plurality of inclination directions; and identifying one inclinationdirection based on each counting result.
 7. The image processing methodaccording to claim 5, wherein said inclination direction detection stepidentifies said four vertex coordinates by: hypothesizing as to aplurality of inclination directions of said rectangular area based onthe plurality of vertex coordinate candidates for the rectangular area;counting the numbers of black pixels existing inside the rectangulararea indicated by each of the plurality of inclination directions; andidentifying one inclination direction based on each counting result. 8.The image processing method according to claim 5, further having a biasfilling correction step of performing bias filling correction to therectangular area determined by the four vertex coordinates detected bysaid inclination detection step.
 9. The image processing methodaccording to claim 8, wherein interpolation by a bilinear method isperformed in said bias filling correction step.
 10. The image processingmethod according to claim 8, wherein interpolation by a nearest neighbormethod is performed in said bias filling correction step.
 11. The imageprocessing method according to claim 1, wherein said rectangle extentdetection step performs trapezoidal approximation to said histogram onthe hypothesis that said rectangular area is a rectangle.
 12. The imageprocessing method according to claim 11, wherein said rectangular areais an area for showing a photo image.
 13. The image processing methodaccording to claim 1, further having a binarization step of binarizing amultivalic original image so as to obtain said binary image.
 14. Theimage processing method according to claim 13, wherein said originalimage is an image having one photo image placed therein.
 15. The imageprocessing method according to claim 13, wherein, in a case in whichsaid original image is an image having a plurality of photo imagesplaced apart in a horizontal scanning direction therein, said histogramcreation step divides said binary image for each of said photo images inthe horizontal scanning direction and creates said histogram for each ofthe divided binary image.
 16. The image processing method according toclaim 13, wherein, in a case in which said original image is an imagehaving a plurality of photo images placed apart in a vertical scanningdirection therein, said histogram creation step divides said binaryimage for each of said photo images in the vertical scanning directionand creates said histogram for each of the divided binary image.
 17. Theimage processing method according to claim 1, further comprising alabeling step of labeling said binary image, and wherein said histogramcreation step creates the histogram having the number of said labeledpixels.
 18. The image processing method according to claim 17, whereinsaid labeling step comprises: a black block detection step of, in saidbinary image, detecting as a black block a block of a predetermined sizehaving a number of black pixels therein equal to or higher than apredetermined ratio; and a label replacement step of replacing the blackpixels inside and connected to the detected black block with apredetermined label value.
 19. The image processing method according toclaim 18, wherein said black block detection step detects as said blackblock a block of which the inside is composed entirely of black pixels.20. The image processing method according to claim 18, wherein saidblack block detection step comprises: a division step of dividing saidentire binary image into said blocks of the predetermined size; and adetermination step of determining whether or not each of said blocks isthe black block.
 21. The image processing method according to claim 18,wherein said black block detection step detects the black block bysequentially deviating the positions of said blocks of the predeterminedsize in said binary image.
 22. The image processing method according toclaim 18, wherein, in the case in which said black blocks of thepredetermined size are not discovered in said binary image, said blackblock detection step detects the black block of a smaller size than thepredetermined size.
 23. The image processing method according to claim18, wherein said label replacement step comprises: a reference labelreplacement step of replacing a reference line inside said black blockwith said label value; an upper part label replacement step of replacingthe black pixels connected to a portion more upward than said referenceline with said label value; and a lower part label replacement step ofreplacing the black pixels connected to a portion more downward thansaid reference line with said label value.
 24. The image processingmethod according to claim 18, wherein said black block is a rectangle.25. The image processing method according to claim 18, wherein saidblack block is a polygon.
 26. The image processing method according toclaim 18, wherein said black block is a circle.
 27. An image processingapparatus for detecting a rectangular area from an image, comprising: ahistogram creation component for creating histograms of the numbers ofhorizontal and vertical black pixels in a binary image made bybinarizing an original image; a rectangle extent detection component forperforming trapezoidal approximation to the histograms so as to detectan extent of the rectangular area in said binary image; and aninclination direction detection component for detecting an inclinationdirection of said rectangular area in said extent.
 28. An imageprocessing program for detecting a rectangular area from an image,comprising: a code of a histogram creation step creating histograms ofthe numbers of horizontal and vertical black pixels in a binary imagemade by binarizing an original image; a code of a rectangle extentdetection step of performing trapezoidal approximation to the histogramsso as to detect an extent of the rectangular area in said binary image;and a code of an inclination direction detection step of detecting aninclination direction of said rectangular area in said extent.
 29. Arecording medium on which the program according to claim 28 is recorded.30. An image processing method for labeling an object in a binary image,comprising: a black block detection step of, in said binary image,detecting as a black block a block of a predetermined size having anumber of black pixels therein equal to or higher than a predeterminedratio; and a label replacement step of replacing the black pixels insideand connected to the detected black block with a predetermined labelvalue.
 31. The image processing method according to claim 30, whereinsaid black block detection step detects as said black block the block ofwhich the inside is composed entirely of black pixels.
 32. The imageprocessing method according to claim 30, wherein said black blockdetection step comprises: a division step of dividing said entire binaryimage into said blocks of the predetermined size; and a determinationstep of determining whether or not each of said blocks is the blackblock.
 33. The image processing method according to claim 30, whereinsaid black block detection step detects the black block by sequentiallydeviating the positions of said blocks of the predetermined size in saidbinary image.
 34. The image processing method according to claim 30,wherein, in a case in which said black blocks of the predetermined sizeare not discovered in said binary image, said black block detection stepdetects the black block of a smaller size than the predetermined size.35. The image processing method according to claim 30, wherein saidlabel replacement step comprises: a reference label replacement step ofreplacing a reference line inside said black block with said labelvalue; an upper part label replacement step of replacing the blackpixels connected to a portion more upward than said reference line withsaid label value; and a lower part label replacement step of replacingthe black pixels connected to a portion more downward than saidreference line with said label value.
 36. The image processing methodaccording to claim 30, wherein said black block is a rectangle.
 37. Theimage processing method according to claim 30, wherein said black blockis a polygon.
 38. The image processing method according to claim 30,wherein said black block is a circle.
 39. An image processing apparatusfor labeling an object in a binary image, comprising: a black blockdetection component for, in said binary image, detecting as a blackblock a block of a predetermined size having a number of black pixelstherein equal to or higher than a predetermined ratio; and a labelreplacement component for replacing the black pixels inside andconnected to the detected black block with a predetermined label value.40. A program for implementing the image processing method according toclaim 30 by being executed on a computer.
 41. A recording medium onwhich the program according to claim 40 is recorded.
 42. An imageprocessing method for trimming an end of a rectangular image,comprising: a histogram generation step of generating a histogram of thenumber of white pixels in a binary rectangular image; a mixed areadetermination step of determining a white pixel mixed area at an end ofsaid rectangular image based on said histogram; and an elimination stepof eliminating said white pixel mixed area from the end of saidrectangular image.
 43. The image processing method according to claim42, wherein said histogram generation step generates said histogram forthe end of said rectangular image.
 44. The image processing methodaccording to claim 43, wherein the end of said rectangular image is anarea comprised of a predetermined ratio of the rectangular image size.45. The image processing method according to claim 42, furthercomprising a binarization step of binarizing a multivalic rectangularimage so as to obtain said binary rectangular image.
 46. The imageprocessing method according to claim 45, wherein said binarization stepsets a binarization threshold based on an instruction from an operator.47. The image processing method according to claim 42, furthercomprising a rectangular image detection step of detecting saidrectangular image from an image of a larger size.
 48. The imageprocessing method according to claim 47, wherein said rectangular imagedetection step detects said rectangular image by automaticallydetermining it.
 49. The image processing method according to claim 48,wherein said rectangular image detection step detects said rectangularimage by: in an image made by binarizing an original image, creatinghistograms of the numbers of horizontal and vertical black pixelsthereof; performing trapezoidal approximation to the histograms andthereby detecting an extent of the rectangular area in said binaryimage; and detecting an inclination direction of said rectangular areain said extent.
 50. The image processing method according to claim 47,wherein said rectangular image detection step detects said rectangularimage based on an instruction from an operator.
 51. The image processingmethod according to claim 50, wherein said rectangular image detectionstep detects said rectangular image by: reading an original image at afirst resolution and thereby obtaining a preview image; displaying saidpreview image; specifying scan coordinates of a rectangle as to saiddisplayed preview image; and reading the rectangular area indicated bysaid scan coordinates in said original image at a second resolutionhigher than said first resolution so as to detect said rectangularimage.
 52. The image processing method according to claim 47, whereinsaid rectangular image is a photo image.
 53. The image processing methodaccording to claim 42, wherein said mixed area determination stepdetermines an area of which the height of said histogram is equal to ormore than a predetermined threshold to be a white pixel mixed area. 54.The image processing method according to claim 53, wherein saidthreshold is set according to the size of said rectangular image. 55.The image processing method according to claim 53, wherein saidthreshold is set based on instruction from an operator.
 56. An imageprocessing apparatus for trimming an end of a rectangular image,comprising: a histogram generation component for generating a histogramof the number of white pixels in a binary rectangular image; a mixedarea determination component for determining a white pixel mixed area atan end of said rectangular image based on said histogram; and anelimination component for eliminating said white pixel mixed area fromthe end of said rectangular image.
 57. An image processing program fortrimming an end of a rectangular image, comprising: a code of ahistogram generation step of generating a histogram of the number ofwhite pixels in a binary rectangular image; a code of a mixed areadetermination step of determining a white pixel mixed area at an end ofsaid rectangular image based on said histogram; and a code of anelimination step of eliminating said white pixel mixed area from the endof said rectangular image.
 58. A recording medium on which the programaccording to claim 57 is recorded.